Chore: Historical chunks#7
Merged
Merged
Conversation
There was a problem hiding this comment.
Pull request overview
This PR adds a new pkg/component-level Reporter for emitting bounded “historical chunk” lifecycle state (progress/completion/verification/failure) and component heartbeats to the flowctl control plane, along with documentation and dependency updates needed to support the new reporting flow.
Changes:
- Added
pkg/component/reporter.goimplementing env-driven configuration, registration, heartbeats, and chunk run upserts. - Added unit tests for env parsing and no-op/required-parameter behavior in
pkg/component/reporter_test.go. - Updated
README.md,go.mod, andgo.sumto document and pull in the required flowctl/proto dependencies.
Reviewed changes
Copilot reviewed 4 out of 5 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| README.md | Documents “Historical Chunk Reporting” usage and env contract. |
| pkg/component/reporter.go | New reporter implementation for registration/heartbeats/chunk state upserts. |
| pkg/component/reporter_test.go | Unit tests for configuration parsing and disabled/required-parameter behavior. |
| go.mod | Adds github.com/withobsrvr/flowctl and bumps flow-proto, plus indirect deps. |
| go.sum | Lockfile updates for newly added/updated dependencies. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+60
to
+64
| cfg Config | ||
| conn *grpc.ClientConn | ||
| client flowctlpb.ControlPlaneClient | ||
| serviceID string | ||
| } |
Comment on lines
+131
to
+135
| if resp.ServiceId != "" { | ||
| r.serviceID = resp.ServiceId | ||
| } else { | ||
| r.serviceID = r.cfg.ComponentID | ||
| } |
Comment on lines
+144
to
+147
| serviceID := r.serviceID | ||
| if serviceID == "" { | ||
| serviceID = r.cfg.ComponentID | ||
| } |
Comment on lines
+75
to
+78
| if cfg.ComponentID == "" { | ||
| return nil, fmt.Errorf("FLOWCTL_COMPONENT_ID is required when ENABLE_FLOWCTL=true") | ||
| } | ||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces support for historical chunk reporting to the SDK, allowing components to report the progress, completion, verification, and failure of bounded ledger work-units to the flowctl control plane. It adds a new
Reporterimplementation, updates documentation, and brings in new and updated dependencies to support these features.Historical Chunk Reporting:
Reporterinpkg/component/reporter.gothat enables components to emit lifecycle and historical chunk state to flowctl, including progress, completion, verification, and failure reporting. This includes environment-based configuration, chunk state updates, heartbeats, and no-op handling when disabled.pkg/component/reporter_test.go, verifying configuration parsing, no-op behavior, and required parameters.Documentation:
README.mdwith a new section on "Historical Chunk Reporting," including usage examples and details on the environment contract. [1] [2]Dependency Management:
go.modto addgithub.com/withobsrvr/flowctland updategithub.com/withObsrvr/flow-prototo v0.1.3, as well as to add and update several indirect dependencies required for chunk reporting and observability. [1] [2]